Entities

• rhq_bundle_type/BundleType: This entity defines a bundle type/format. We will initially support one primary bundle type: Ant. Additionally we will support the FileTemplate bundle type for simple scripting and testing. A Puppet bundle type is being considered. Plugins on both the Server and Agent side will check a bundle's type to determine how to parse and validate that bundle's recipe file. Each bundle type is associated with a singleton ResourceType whose ResourceComponent implements the BundleFacet, which contains the logic for checking and installing (if needed) a recipe.

• rhq_bundle/Bundle: This is an umbrella entity describing the bundle content and acting as a root for the related entities. Bundles are versioned, so the real data is stored in rhq_bundle_version

• rhq_bundle_version/BundleVersion: This entity is a specific instance of the bundle: the bundle distribution file, recipe and or bundle files will differ between versions. A bundle version is what is physically deployed to a destination platform/directory.

• rhq_bundle_file/BundleFile: This entity is a file which is part of a bundle and is used in the deployment process. A bundle distribution file contains the recipe and 0 or more of teh referenced bundle files.

• rhq_bundle_destination/BundleDestination: An entity describing to where a bundle will be deployed. It refers to a resource group of platforms and a root deploy directory.

• rhq_bundle_deployment/BundleDeployment: An entity representing a physical deployment. Defines the bundle version being deployed, the destination it will be deployed to, and the configuration used for the deployment. The configuration defines the variable substitutions needed, as defined by the config_def defined for the relevant bundle_version.

• rhq_bundle_res_deploy/BundleResourceDeployment: This entity represents a physical deployment of the bundle onto a given resource.

• rhq_bundle_res_dep_hist/BundleResourceDeploymentHistory: This entity represents an audit trail for the deployment of the bundle onto a given resource (a platform). It will (minimally) record start and end times for deployment but will typically contain robust deployment auditing.

• rhq_tagging: This entity represents a tagging relationship users can use to reflect their provisioning processes. (Best practices to be published for this?)

Relationships

The following relationships are shown on this design:

• (a): The resource group for a destination must contain only Platform resources as explicit group members.

• (b): Each bundle will have its own repo for owning content associated with the bundle that is not pulled from an existing repo. Note, the repo in this relationship is completely independent of those described in (a).

• (c): Each bundle will have its own package type. Files uploaded for the bundle will have their package types set to the bundle's package type and the package versions will be associated with the bundle's repo (see (b)).

• (d): Bundles are versioned. This is the version relationship.

• (e): Bundles are not hierarchical. Therefore, a user needs to be able to deploy more that one bundle to a given platform.

• (f): A bundle may require that a target resource be subscribed to several repositories for content. For example, this bundle requires the machine to be subscribed to a RHEL6 repo. This relationship models that need. Only eligible platforms can be provisioned to.

New Bundle Creation

1. The user does development work on a bundle, and combines them all into a single archive file.

2. The user uploads the file and notes the "type" of bundle (and, file, etc)

3. The Bundle Subsystem locates a custom Bundle Parser (via a bundle server-side plugin) based on the type which will expand the archive file and create the new data structure items in the database.

Bundles and Bundle Versions

A bundle is a named collection of content and configuration. A bundle can be versioned. Each version can contain the following bits of information:

1. The repositories which a machine should be subscribed to.

2. The type of bundle which it is (e.g. FileTemplate, Puppet, etc)

3. An optional recipe

   1. The recipe content will be stored in an instance of rhq_package_version.

   2. The recipe should not change for a bundle version - if a recipe changes, it must be considered a new bundle version - therefore, the recipe must be hardlinked to its bundle_version - the recipe content will be stored as a CLOB in the bundle_version table.

4. An Collection of template variables which need to be filled out by a deployment.

   1. Stored in configuration definitions / configuration templates

5. A collection of default values for the template variables

6. A version number, and last modified date to help determine if an update needs to be run.

7. An md5 sum and last modified data for the recipe.

Bundle Files

Bundle files represent the actual files which should be included in the bundle. When creating a bundle file, the user can decide to either pull in the latest version of the file, or to pull in an exact version of the file. Base on this selection, the entity will  contain a reference to either a rhq_package or to a rhq_package_version.

Deployments

The deployment should be reusable, so the deployment is broken up into 2 entities. These are:

• rhq_bundle_deploy: This entity will hold the user defined values which are required to complete the deployment. For example, the templated file has the ability to substitute in a "wiki_name" variable, then his object will contain a key-value pair which is wiki_name = "My Wiki". This is done via the  rhq_config Configuration subsystem.

• rhq_bundle_res_deploy: This entity provides a mapping between one deployment and many systems. For each platform where a bundle is deployed, there will be one rhq_bundle_res_deploy entity.

The Deployments, when mapped to a resource, they will need to be ordered. The user will want to ensure that certain bundles are deployed before or after others.

Agent Plugin Container

The Agent Plugin Container's Bundle Manager (org.rhq.core.pc.bundle.BundleManager) will support the bundle subsystem on the agent side. It will interface with those plugins that support the BundleFacet.

Agent / Server Communication

The Agent / Server communication will follow the standard format for RHQ subsystems. The specific APIs are below.

Bundle DSL

The Bundle contains a single script which is the set of actions or "recipe commands" to be taken on a given machine in order to deploy the Bundle. The DSL will consist of the following recipe commands:

• script: Takes as a parameter a command line which is executed on the machine. This represents a script file that is explicitly included in the bundle - it is an uploaded bundle file.

• command: Takes as a parameter a shell command line which is executed on the machine. The executable/shell command must already exist on the machine.

• deploy: Takes 2 parameters, an archive name (-file) and a path (-directory). The archive is assumed to be one of the bundle_files. At deployment time, the archive is unzipped into the path location. The MD5 version of the file will need to be stored on the machine to ensure that the a second deployment is not done unless the version changes.

• file: Takes in 2 parameters, a file name (-source) and a destination path (-destination). The file is assumed to be one of the bundle files. At deployment time, the file is copied in the specified path. If you want replacement variables to be substituted with their realized values in this file, use the realize command afterwards.

• realize: Realize perform macro substitution on a file which is already on the filesystem (--file). This can be used to substitution replacement variables.

• configdef: Defines one configuration definition, or "replacement variable". The name provided to this configdef will be assumed to be a replacement variable that may exist in bundle files that are to be realized. The user that deploys the bundle will need to provide a value for this config def or let the agent provide the value (by looking up the value in the agent's template engine or looking up the value as a simple Java system property).

• service: Takes in 2 parameters, a service name and a command (start|stop|restart).

• repo: Takes as a parameter a repository name so the server will be able to check that the proper subscriptions are in order prior to deploying the bundle.

• package: Takes as a parameter a packge name. It will cause a specific package to be installed or updated.

An example DSL would then be:

  repo -n rhel-x86_64-5 (_not supported yet_)
  package -n foo-1.25.rpm (_not supported yet_)
  package -n bar-1.25.rpm (_not supported yet_)
  configdef -n listener.port
  script foo.bash -c some parameter
  deploy -f jboss.tar -d <%jboss.home.directory%>
  realize -f <%jboss.home.directory%>/server/default/setting.xml
  file -s example.setting -d /etc/some/setting.ini
  service example restart (_not supported yet_)

There must be a server-side plugin that must be defined that can parse the different kinds of recipes that are supported. For example, a file-template server side plugin must be able to parse a file-template DSL as shown above so it can:

• create the bundle version config def so a UI will know what questions to ask a user

• create relationships between the bundle version and its related repos, distributions and packages

Server-side SLSB

• BundleManagerBean should support the following API for CLI/GUI usage

• RecipeParseResults verifyRecipe(String bundleTypeName, String recipe)

• Bundle createBundle(String bundleName)

  • Creates the Repo under the covers

• BundleVersion createBundleVersion(Bundle bundle, String Recipe)

  • Implicit package creation for each bundle file in recipe

  • Implicit create of config def based on recipe

  • Auto-increment version

• List<Package> getBundleVersionPackages( BundleVersion bundleVersion )

  • note, maybe get this as fetch from BundleVersion criteria?

• BundleFile addBundleFile( BundleVersion bundleVersion, PackageVersion pkgVersionPojo, boolean pinToPackage )

  • Calls content subsystem's "createPackageVersion" under the covers

• void attachPackageBitsToBundleFile(BundleFile bf, byte[] bits)

• void attachPackageBitsToBundleFile(BundleFile bf, InputStream bits)

• Convienence method to combine the addBundleFile and attachPackageBitsToBundleFile:

  • BundleFile addBundleFile( BundleVersion bundleVersion, PackageVersion pkgVersion,boolean pinToPackage, byte[] bits )

  • BundleFile addBundleFile( BundleVersion bundleVersion, PackageVersion pkgVersion,boolean pinToPackage, InputStream bits )

• BundleDeployDefinition createBundleDeployDefinition(BundleDeployDefinition def)

• void deployBundle(int bundleDeployDefinitionId, int[] resourceIds)

  • Question: Does this generate the BundleDeployment records (the mapping table) Answer: Probably, although it may not be needed until the deployment actually starts (if it is scheduled for the future). History records hang off this and get generated on each tracked action.

• List<BundleDeploymentHistory> getBundleDeploymentHistory(BundleDeploy deploy)

• List<BundleDeploymentHistory> findBundleDeploymentHistoryByCriteria(BundleDeploymentHistoryCriteria bc)

  • bc.addFilterBundleId( id )

• List<BundleDeploymentHistory> getBundleDeploymentHistory(int bundleId,int platformResourceId);

  • gets history for a specific bundle deployment

• List<BundleDeploymentHistory> getBundleDeploymentHistory(int platformResourceId);

  • gets history for ALL bundles deployed on this platform

Server-side plugin

• interface BundleServerPluginFacet is what all bundle server side plugins must implement. We need to fill out this interface with the API described in this section

• RecipeParseResults parseRecipe(String recipe)

• class RecipeParseResults needs to contain all information that can be gleened from the recipe:

  • ConfigurationDefinition configurationDefinitionAndTemplates

  • Do we need a separate ConfigurationTemplate? or can they be in ConfigurationDefinition?

  • List<Package> bundleFiles

Agent-side Plugin Container/Manager

• interface BundleAgentService (and the impl BundleManager) needs to support the API in this section

• BundleDeployResults deployBundle(BundleDeployRequest request)

  • request has to have things like "getBundleDeployDefinition"

Agent-side bundle plugin

• all agent bundle plugins must implement the BundleFacet. This facet should support the API described in this section

• BundleDeployResults deployBundle(BundleDeployRequest request)

  • Note: plugins never get resource IDs, nor to they typically get the detailed data that server sends to agents (for several reasons - don't forget we want the plugin dev API to be simple). Thus I think the BundleFacet probably won't be a one-to-one mapping of the BundleAgentService interface - e.g. plugin dev won't need to know about BundleDeployResults/Request but a more simple API.

  • The package org.rhq.core.clientapi.[server,agent].bundle will have interfaces and DTOs for agent-server comm APIs - NO agent plugin interfaces/DTOs go in here. Plugin devs do not even have access to client-api classes.

  • The package org.rhq.core.pluginapi.bundle has the plugin developer API such as BundleFacet and all its supporting objects. The server side does not have access to these classes - its only for agent side plugin container-plugin communication.

Properties Available

In the file template recipe, you can provide <% var %> replacement variables, where "var" is one of the below. For an ANT recipe, the below properties will be automatically set and can be used as a ${var} expression:

These have the values as specified by their corresponding SystemInfo API:

These have the values of their corresponding standard Java System Property:

• rhq.system.sysprop.java.io.tmpdir

• rhq.system.sysprop.file.separator

• rhq.system.sysprop.line.separator

• rhq.system.sysprop.path.separator

• rhq.system.sysprop.java.home

• rhq.system.sysprop.java.version

• rhq.system.sysprop.user.timezone

• rhq.system.sysprop.user.region

• rhq.system.sysprop.user.country

• rhq.system.sysprop.user.language